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(g) Image encoding and decoding method and apparatus using edge systhesis and inverse wavelet 
transform. 



@ A digitized image Is encoded by detecting 
edges in the image, encoding the position and 
sharpness of the detected edges, filtering the 
image by a low-pass filter to generate a low- 
frequency image, and encoding the . low-fire- 
quency image. A digitized image encoded in 
this way is reconstmcted by generating a hori- 
zontal edge image and a vertical edge image 
from the encoded edge position and sharpness 
information, synthesizing a pair of high-fre- 
quency images by filtering the horizontal and 
vertical edge images with an edge synthesis 
filter, decoding the low-frequency Image, and 
performing an inverse wavelet transfonm on the 
decoded low-frequency image and the high- 
frequency images. 
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BACKGROUND OF THE INVENTION 

This invention relates to a compressive image encoding and decoding method using edge synthesis and 
the inverse wavelet transform, and to digital image encoding and decoding devices employing this method. 

5 Compression is essential for efficient storage and transmission of digitized inrtages. Compression methods 

have been described by the Joint Photographic Experts Group (JPEG) for still images, and the Motion Picture 
Experts Group (MPEG) for nnoving Images. The JPEG method involves a discrete cosine transform (OCT), 
followed by quantization and variatrfe-length encoding. The MPEG method involves detecting motion vectors. 
Both methods require substantial computation, the detection of motion vectors being particularly demanding. 

to Recently there has been much Interest in the wavelet transform as a means of obtaining high cbmpressipn 

ratios with relatively modest amounts of computation. This transform employs a family of wavelets related by 
dilation and translation; that is, the family consists of occurrences of the same basic vyavelet at different lo- 
cations and on different scales. If the scales form a progressively doubling sequence, and If the basic wavelet 
is zero everywhere except in a limited domain, wavelet transforms and inverse wiaivelet transforms can be car- 
ts ried out with efficient computational algorithms. 

A wavelet transform can be described as a filtering process executed at each wavelet scale. A digitized 
image, for example, is transformed by filtering with the basic wavelet, then with the basic wavelet dilated by 
a factor of two, then with the basic wavelet dilated by a factor of four, and so on. 

One prior-art wavelet encoding scheme employs a complementary pair of wavelets to divide an image into 

20 a high-frequency component and a Ibw^frequency component. These components contain information about 
variations on scales respectively less than and greater than a certain cut-off scale. This process is iterated on 
the low-frequency component with a doubling of the wavelet scale, obtaining new low^f requency and high- 
frequency components, then iterated again on the new low-frequency component, and so oh. After a certain 
number of iterations, the components are encoded by an encoding scheme that works from low-toward high- 

25 frequency information. This scheme enables accurate image reconstruction, but retains too much high- 
frequency infonTiation to achieve high compression ratios. 

Another prior-art wavelet encoding scheme employs a basic wavelet that is the first derivative of a smooth- 
ing filter (that is, the first derivative of a low-pass fjltering function). This type of wavelet acts as a high-pass 
filter. High-frequency infonrnatiori is obtained by detecting local peaks (local rnaxima of absolute values) in the 

30 result of the wavelet transform, which conrespond to edges in the original image. The size and location of the 
peak values at a selected scale are encoded, along with a low-frequency image obtained by smoothing at the 
largest scale of the wavelet transform. Fairly high compression ratios can be obtained in this vvay. . 

To reconstruct the original image from the encoded data, this prior-art method employs an algorithm de- 
rived from a mathematical procedure involving iterated projections in Hilbert space. Under ideal conditions, 

35 the projections converge toward a unique set of data that (i) have the required local peak values and (ii) are 
within the range of the wavelet transform operator. An inverse wavelet transform is then earned but on the con- 
verged data to obtain the original image. 

It has yet to be shown, however, that the projections always converge, or that data satisfying conditions 
(i) and (ii) are unique. In practice, there is difficulty in knowing when to stop iterating. For some images, it seems 

40 that data satisfying (i) and (ii) are not unique, and instead of converging, the iteration wanders endlessly through 
Hilbert space, first approaching the desired image transform, then moving away again/ 

Still higher compression ratios have been obtained by a fractal compression scheme, but this requires a 
very large amount of computation in the encoding process. 

45 SUMMARY OF THE INVENTION 

It is accordingly an object of the present invention to encode digitized images eff rciently, with a high com- 
pression ratio but without requiring extensive computation. 

Another object of the invention is to encode moving images efficiently. 
50 still another object is to reconstruct digitized images from encoded information efficiently and reliably. 

Yet another object is to reconstruct digitized moving images efficiently and reliably. 
The invented method of encoding a digitized image comprises the steps of: 
detecting edges in the digitized image; 

encoding the position and sharpness of the detected edges to generate edge information; 
55 filtering the digitized image by a low-pass filter to generate a low-frequency image; and 

encoding the low-frequency image to generate low-frequency information. 
The invented method of reconstructing a digitized image encoded by the invented encoding method com- 
prises the steps of: 
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generating a horizontal edge image and a vertical edge image from the encoded edge information; 
synthesizing a pair of high-frequency images by filtering the horizontal edge image and vertical edge 
image with an edge synthesis filter, 

decoding the encoded low-frequency information to obtain a tow-frequency image; and 

performing an inverse wavelet transform on the low-frequency image and the high-frequency images. 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a digital image encoder in a first emboctii^ient of the invention. 
10 FIG. 2 is a block diagram of a digital image decoder in the first embodiment 

V FIG. 3 is a graph illustrating pixer values at an edge. 

FIG. 4 is a graph illustrating pixel values at another edge. 
FIG. 5 is a graph illustrating pixels that do not form an edge. 

FIG. 6 is a block diagram of the edge synthesizer in a second embodiment of the invention. 
15 FIG. 7 is a block diagram of the digital image encoder in a third embodiment of the invention. 

FIG. 8 is a block diagram of the digital image decoder in the third emtx>diment. 
FIG. 9 is a block diagram of the digital image encoder in a fourth embodiment. 
. FIG. 10 is a block diagram of the digital image encoder in a fifth embodiment. 
FIG. 11 is a block diagram of the digital image decoder in the fifth embodiment 
20 FIG. 12 is a block diagram of the digital image encoder in a sixth emt>odiment 

FIG. 13 is a block diagram of the digital image encoder in a seventh embodiment, for encoding moving 
images. 

FIG. 14 is a block diagram of the digital image decoder in the seventh embodiment. 
FIG. 15 is a block diagram of the digital image encoder in an eighth embodiment 
25 FIG. 16 is a block diagram of the digital image encoder in a ninth emt>odiment 

FIG. 17 is a block diagram of the digital image encoder in a tenth eml>odiment 
FIG. 18 is a block diagram of the digital image decoder in the tenth embodiment 
FiG. 19 is a block diagram of the digital image encoder in an eleventh embodiment. 
FIG. 20 is a block diagram of the digital image encoder in a twelfth embodiment 
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DETAILED DESCRIPTION OF THE INVENTION 



Embodihients of the invention will be now described with reference to the attached illustrative drawings, 
starting with the simplest case in which the inverse wavelet transform is limited to a single scale. The following 
35 terminology will be used. 

A (two-dimensional) digitized image is an array of pixels having values x(i, j), where the horizontal coor- 
dinate i and vertical coordinate j range over sets of integers. The range of these coordinates is the size of the 
image, i.e. the number of pixels in the horizontal and vertical directions. The pixel values represent, for example, 
- intensity gradations. 

40 A one-dimensional image is a horizontal or vertical line in a two-dimensional digitized image, i.e. a set of 

pixels with values x(i) indexed by a single coordinate. 

A standard edge is a one-dimensional image having just one sharp variation point An example would be 
an image with pixel values that change first at one constant rate, then at a different constant rate, siich as the 
following: 

45 ...2.0 1.5 1.0 0.5 0 0.5 1.0 1.6 2.0... 

In this standard edge the pixel values first decrease at a rate of -0.5 per pixel, then increase at a rate of 
+0.5 per pixel. Other standard edges of the same general type are shown in FIGs. 3 and 4. A standard edge 
serves as a basic model for all the edges occurring in a digitized image. 

Down-sampling a digitized image means reducing its size by a process generally analogous to photograph- 
50 ic reduction. Up-sampling means increasing the size of an image by a process generally analogous to photo- 
graphic enlargement In up- or down-sampling the entire image is enlarged or reduced, its features increasing 
or decreasing in scale. 

A (one-dimensional) filter is a set of coefficients f(k), where k ranges over the integers. The non-zero coef- 
ficients f(k) are the taps of the filter. 
55 The Fourier transform F of a fitter f is defined as follows, where summation is over all taps, e is the natural 

logarithm base, and j is the square root of minus one. 
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F{ w ) = I f(k)-e-J 
k 

The conjugate f of a f liter f is obtained by reversing the order of coefficients: 

f*(k) = f(-k) 

The Fourier transform of f is therefore the cornplex conjugate of the Fourier transform of f. 

A filter has even symmetry around Iq if it has equal values on both sides of Iq : that is, for all integers k: 

f(io - k) = f(lo + k) \ 
In particular, a filter has even symmetry around zero if it Is equal to its own conjugate. 
A filter has odd symmetry around io if it has opposite values on both sides of io, that is: 

f(io - k) = -f(io + k) . . . 

Afilter satisfies the exact reconstruction condition if its fourier transform F satisfies the following condition 
for all values of 0) : 

|F(©)|2 + |F(cd + ii)|2 = 1 
Two filters with Fourier transforms G and H satisfy the exact reconstruction conditbn if: 

|G(G))|2 + |H(cd)|2 = 1 

Filters satisfying these exact reconstruction conditions, also referred to as perfect reconstruction condi- 
tions, are well known in the wavelet transform art A pair of filters satisfying the second exact reconstruction 
condition above is referred to as a complementary pair. 

To filter a one-dimensional image x by a filter f means to obtain a new one-dimensional Image y with the 
following pixel intensities: 

y(i) = S x(i + k) vf (k) 

This operation is equivalent to the conventional convolution operation using the conjugate filter f*. 

yd) = X x(l - k) f*ik) 
k 

The difference between convolution and filtering, as the term is used herein, is purely one of notation. The 
35 meaning of the following description and claims would be unchanged if the terms convolution and convolve 
were used throughout. 

To filter a digitized image x horizontally by a filter f means to obtain a new image y as follows : 

40 y(i» J) = ^ x(l + k. j)-f(k) 

k 

Similarly, f iltering image X vertically by f means: . 

y(i. J) = X x(l. J ♦ k)-t(k) 

k . 

Filtering can be done two-dimensk>nally by, for example, filtering first in the horizontal direction, then in 
50 the vertical direction, or vice versa. . 

A high-pass filter is a filter that retains small-scaie variations and rejects large-scale variations. A low^pass 
filter rejects small-scale variations and retains large-scale variations. Smoothing filter is a synonym for low- 
pass filter. 

The cut-off frequency of a high-pass or low-pass filter indicates the scale that divides the retained varia- 
55 tions from the rejected variations. A lower cut-off frequency corresponds to a larger scale of variations. Fre- 
quency actually refers to the variable o in the Fourier transform F(cd) of the filter. 

Generally speaking, the Fourier transform of a high-pass filter satisfies F(0) = 0, while for a low-pass filter, 
I F(0) I > 0. If I F(0) I = 1 , a low-pass filter can retain large-scale variations without changing their range of 
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variation. Incidentally, F(0) is the sum of the fitter coefficients f(k). 
First ennbodiment 

5 Referring to FIG. 1, the digital image encoder in the first embodiment comprises an input terminal 10, 

smoothing filter 20, edge detector 21, encoding section 22, and down sampler 23. The encoding section 22 
comprises an edge Image encoder 24, low-frequency image encodier 25, and multiplexer 26. 

A digitized image Xq is Input from the input terminal 10 to both the smoothing filter 20 and edge detector 
21 . The smoothing filter 20 outputs a low-frequency image to the down sampler 23, which outputs a reduced 
10 image R to the low-frequency image encoder 25 in the encoding section 22. The low-frequency image encoder 
25 encodes the reduced image R to obtain tow-frequency information Cr. The edge detector 21 detects edges 
in the input image Xq and outputs horizontal and vertical edge images Sh and Sv to the edge image encoder 
24 in the encoding section 22. The edge Image encoder 24 encodes these edge images to obtain edge infor- 
mation Cs. The multiplexer 26 combines the low-frequency information Cr and edge information Cs into an 
15 encoded image C, which is output to a first input/output device 27 such as a communication channel, semi- 
conductor memory, or magnetic disk storage device. 

Referring to FIG. 2, the digital image decoder has a decoding section 28 comprising a demultiplexer 30, 
. edge information decoder 31 , and low-frequency information decoder 32. The digital image decoder also has 
:t. an edge synthesizer 34, up sampler 35, and inverse wavelet transform processor 36. 
20 The demultiplexer 30 receives the encoded image C from the first input/output device 27 and separates 

it into edge information Cs, which it supplies to the edge information decoder 31 , and low-frequency information 
Cr, which it supplies to the low-frequency information decoder 32. The edge information decoder 31 decodes 
Cs to obtain the edge images Sh and Sv, from which the edge synthesizer 34 synthesizes horizontal and vert- 
ical high-frequency images Yh and Yv. The low-frequency information decoder 32 decodes Cr to obtain a de- 
25 coded reduced image R', which the up sampler 35 enlarges to obtain a decoded low-frequency image Xi*. The 
inverse wavelet transform processor 36 receives the high-frequency images Yh and Yv from the edge syn- 
thesizer 34 and the low-frequency image Xi' from the up sampler 35, carries out an inverse wavelet transform, 
and outputs a reconstructed image Xo* to an input/output device 37 such as a video display unit or printer. 
Each of the elements in the FIGs. 1 and 2 comprises a memory for storing data and well-known circuits 
30 for performing arithmetic and logic operations. Descriptions of the circuit configurations will be omitted to avoid 
obscuring the invention with irrelevant detail. The entire digital image encoder can be fabricated as a single 
semiconductor integrated circuit, or can be built from a small nurriber of such circuits; likewise the digital image 
decoder. The invention can also be practiced by providing a general-purpose processor such as a microprb^ 
cesser or digital signal processor with memory for storing data and programs for executing the functions of 
35 the individual elements in FIG. 1 or FIG. 2. 

Next the operation will be described, starting with the operatbn of the encoder in FIG. 1. 
The input Image Xo is a digitized image with pixel values Xo(i. j). The smoothing filter 20 is a low-pass filter 
with even symmetry around zero and an odd number of taps. The tap coeff icients will be denoted ho(k), where 
k ranges from -N to N, so the filter has 2N + 1 taps and the even symmetry condition is: 
40 ho(- k) = ho(k) (k = 1. 2, N) 

The low-pass filter ho satisfies the exact reconstruction condition, and the sum of its tap coefficients is 
plus or minus one. That is, its Fourier transform Ho satisfies the following conditions: 

|Ho(tD)|2 + |Ho(fl> + n)|2 = 1 . 
IHo(0)I =1 

45 The smoothing filter 20 performs a two-dimensional low-pass filtering operation by filtering the input im- 

age Xq with ho horizontally and vertically, obtaining: 

N N 

Xi(i, J) = X £ Xo(i + k. J ♦ 1) -hoCk) -hod) 
50 k = -N 1 = -N 



The down sampler 23 down-samples the low-frequency image X^ output by the smoothing filter 20, thereby 
reducing the amount of image data. One simple method of down-sampling is to divide the image into blocks 
55 of M x M pixels each, and replace each block by a representative pixel value such as the mean or median value 
in the block. This reduces the size of the image by a factor of M in both the horizontal and vertical directions, 
and reduces the amount of image data by a factor of M^. The reduced image R output by the down sampler 
23 resembles a photographie reduction of the original inriage Xq. 
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The low-frequency image encoder 25 encodes the reduced image R by, for example, performing a discrete 
cosine transform, quantizing the resulting DCT coefficients, and encoding them in zig-zag order by a variable- 
length encoding method. Alternatively, the reduced image R can be encoded by a predictive encoding method 
similar to the differential pulse-code modulation method (DPCM) commonly employed for audio signals, or by 
5 any other suitable method. If the above value of M is sufficiently large, high coding efficiency is not critically 
important, because the amount of data to be encoded has already been greatly reduced by down-sampling. 

The edge detector 21 detects edges in the input image Xo by detecting variations in the rate of change of 
pixel values from pixel to pixel. More specifically, edge detector 21 takes differences between adjacent pixel 
values, then takes differences between these differences, and finally performs a thresholding operation to 
10 reduce small differences to zero. This process is carried out separately in the horizontal and vertical diirections. 
- ' In the horizontal direction, for each pixel, the edge detector 21 computes the following pixel difference 
Pdh(iJ): 

Pdh(i,j) = Xo(i,j) - Xo(i - l.j) 
Next it computes differences Sh(i, j) of these pixel differences Pdh(i, j) as follows: 

IS 

Sh(i, J) = Pdhd * 1, J) - Pdhd . .j ) 

= Xo(1 + 1. J) - 2Xo(i. j) t Xo(i - 1. J) 

20 Equivalent edge detection could be performed by filtering the Image Xo with a second^derivative filter hav- 

ing coefficients (1, -2, 1), but taking differences of differences is preferable because it requires less compu- 
tation. 

The detected value Sh(t, j) is the edge sharpne;ss at pbcel (i, j), also referred to as the edge size. Edge sharp- 
ness values with absolute values equal to or less than a certain threshold value T are quantized to zero. Values 
25 exceeding T are also quantized, to reduce the number of bits in the edge-image data; then the quantized sharp- 
ness values Sh(i, j) at all pixels are output as the horizontal edge image Sh. Non-zero pixels in the edge image, 
i.e. pixels at which I Sh(i, j) I > T, are referred to as edge points^ 

The meaning of the edge sharpness computation can be seen in FIG. 3. The horizontal axis represents, 
for example, the horizontal direction in the image, and the vertical axis represents the pixel value. The differ- 
30 ence between pixel values 38 and 39 is -A, the difference between pixel values 40 and 38 Is B, and the dif- 
ference between these differences is B ^ (^A) = A+.B. If A+ B > T, then pixel 38 is an edge point with sharpness 
A + B. The other pixels in FIG. 3 have zero sharpness. FIG. 3 thus illustrates a standard edge, with just one 
sharp variation. 

FIG. 4 shows another standard edge, with non-zero sharpness at a single pixel 41 . The size of the sharp- 
35 ness is 0 , the difference to the left of pixel 41 being zero. FIGs. 3 and 4 are drawn so that e = A + B, and 
edge points 38 and 41 have the same sharpness, or size. 

In FIG. 5, pixel 42 is not an edge point. The differences on both sides of this pixel are the same, so the 
edge sharpness is zero. FIG. 5 represents an area in which, for example, intensity is shading from light to dark 
at a regular rate with no sharp change in this rate at any point Incidentally, FIG. 5 represents the difference 
40 between FIGs. 3 and 4. 

The vertical edge image is generated in the same way^ by taking differences in the vertical direction, then 
taking differences of these differences. The edge sharpness Sv in the vertical direction is: 

Sv(iJ) = Xo(i,j + 1) - 2Xo(iJ) + Xo(U - 1) 
The vertical edge image is quantized as was the horizontal edge image, points for which I Sv(i, j) I ^ T 
45 being set to zero. Quantization can be performed by dividing the edge sharpness values by a power of two, 
for example, or by using a quantization table. Incidentally, the same notation Sh and Sv will be employed herein 
to denote edge sharpness values both before and after quantization. 

The edge image encoder 24 encodes the quantized horizontal and vertical edge images Sh and Sv by 
using, for example, run-length encoding to encode the positions of edge points, and differential encoding to 
50 encode the sharpness values at these points. Alternatively, chain encoding can be used to encode the edge 
positions, taking. advantage of the fact that edge points tend to be organized into chains of mutually contiguous 
points. Chain encoding encodes only the displacement from one position to the next in such a chain. 

The encoded image C, consisting of the edge information Cs and low-frequency information Cr as multi- 
plexed by the multiplexer 26, is output to the first input/output device 27 for transmission or storage. The amount 
55 of encoded data is generally reduced to a small fraction of the amount in the original image Xo, so the encoded 
image C can be efficiently transmitted or stored. 

Next the decoding operations will be described. These operations employ further filters e©, f©. and go. 
Filter go is a high-pass filter having even symmetry and 2N + 1 taps, related to the smoothing filter ho as 

6 
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follows: 

go(k) = (-l)'^ho(-k) 

This relation implies that the Fourier transform Go of go is the complex conjugate of Ho(<o + n), so ho and 
go satisfy the exact reconstruction condition and form a complementary pair. Furthermore, Go(cd * n) is the 
5 complex conjugate of H{<o), so Qq also satisfies an exact reconstruction condition by itself: 

IHo(g>)|2 + |Go(cd)|2 = 1 
|Go(o))|2 + |Go(a) + = 1 
Since I Ho(0) I = 1, it follows that I Go(0) I = 0, so the sum of the tap coefficients go(k) is zero. 
The above properties of go imply that when a uniform gradation of the type shown in FIG. 5 is filtered by 
10 . go, the result will be uniformly zero. Consequently, if a standard edge of the type shown in FIG. 3 or 4 is filtered 
z. by go, the result will depend only on the edge sharpness. For example, FIGs. 3 and 4 give the same result when 
filtered by go- Moreover, this result has only 2N - 1 non-zero values, occurring in the vicinity of the edge point. 

Filter eo is an edge synthesis filter derived by filtering a standard edge of unit sharpness (for example, the 
edge in FIG. 4 with e = 1) by go. The result is: 

15 

N 

e(t) = e{-t) - 5: (k - t) •ffo(^) if O < t < N 

k=t 

20 . - 

e(t) = e(-t) = 0 ift^N 
Filter fo is a completk^n filter, defined in terms of Its Fourier transform F© as follows. 

Fo(co) = " 'y°<">'' 

25 Filter fo is obtained by executing an inverse Fourier transform on Fq. Like ho and go, filter fo has even sym- 

metry and 2N + 1 taps. The above definition of Fo, incidentally, is well known in the wavelet transform art and 
comes from the following equation: 

|H(c0)h + 2.F(co) rG(a))|2 = 1 
Referring again to FIG. 2, after the demultiplexer 30 separates the Input encoded image C Into edge in- 

30 formation Cs and low-frequency information Cr, decoders 31 and 32 carry out processes that are reverse to 
the processes performed by encoders 24 and 25 in FIG. 1. For example, if the low-frequency image encoder 
25 in FIG. 1 performed a discrete cosine transform followed by quantization and variable-length encoding, the 
low-frequency information decoder 32 in FIG. 2 performs variable^length decoding, dequanitization, then an 
inverse discrete cosine transform to obtain the decoded reduced image R'. 

35 If the edge images were encoded by run-length or chain encoding of the positions of the edge points and 

differential encoding of their sharpness, the edge information decoder 31 performs run-length or chain decod- 
ing to recover the edge positions and an accumulation process to obtain the sharpness values. The edge in- 
formation decoder 31 also dequantlzes the sharpness values: 

The up sampler 35 obtains the decoded low-frequency image Xi by restoring the decoded reduced image 

40 R' to the original image size (the sb^e of the edge images and high-frequency images). If, for example, each 
pixel value in the reduced image R represented an M x M block in the low-frequency image X^, the up sampler 
. 35 can simply copy this value to all pixels in the corresponding M x M block of the decoded low-frequency image 
Xf*. More sophisticated up>-sampling methods involving interpolation can also be employed, but smooth inter- 
polation is not required, because the image will be smoothed during the inverse wavelet transform. 

45 The edge synthesizer 34 filters the horizontal edge image Sh by the edge synthesis filter e(t) in the hori- 

zontal direction, and the vertical edge image Sv by e(t) in the vertical direction. The resulting horizontal and 
vertical high-frequency images Yh and Yv can be descrit>ed by the following equations: 



50 N-1 

Yh(l, J) = E Sh(l + j)c(t) 
t = l-N 



55 



Yv(l. J) = 



Z Sv(i. J + t)e(t) 
t = l-N 
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Conceptually, for each edge point, the edge synthesizer 34 synthesizes the result of filtering a standard 
edge of the corresponding sharpness by the high-frequency filter go. It then adds these results together for all 
edge points to synthesize a high-frequency image. 

The inverse wavelet transform processor 36 performs three twd-dlmensional filtering operations: it filters 
the decoded low-frequency image X^' by ho horizontally and vertically; fitters the horizontal high-frequency im- 
age Yh by go horizontally and f© vertically; and filters the vertical high-frequency image Yv by go vertically and 
fo horizontally. The three results are added to obtain the reconstructed image Xo' . This inverse wavelet trans- 
form is well known in the art, and is described by the following equation: 

■ ■. ■ ■. ■ ■.■ ■ • ; 

- . - - N N . • : ' - . 
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The reconstructed image Xq' is output to the second input/output device 37 for display or print-oiiL Alter- 
natively, the reconstructed image can be output to a computer for further image processing. 
25 Synthesizing high-frequency image information from edge images as described atx)ve enables high com- 

pression ratios to be obtained. High-frequency information corresponding to edges below the sharpness 
threshold T is tost, so there is some loss of low-level, high-frequency shading variations^ but the reconstructed 
image is not distorted, and sharp edges are faithfully preserved. Moreover, the high-frequency information is 
synthesized in a single, simple filtering step, without recourse to iterative procedures of uncertain reliability. 



Second emt>odiment^ 



The second embodiment is similar to the first, but uses filters with an even instead of an odd number of 
taps, so the high-pass filter has odd symmetry, and the edge synthesis procedure is modified accordingly. 
35 The digital image encoder of the second embodiment is identical to the digital image encoder in FIG. 1, 

except that the smoothing filter 20 employs a filter he with 2N taps. This filter has even symmetry around 1/2: 
. he(1 ^ k) = he(k) (k = 1, 2. .... N) 

The Fourier transform He of this filter satisfies the same conditions as in the.first embodiment 

Ih«(0)I =1 

40 |He(co)|2 + |He(€D + n)|2 = 1 

The smoothing filter 20 obtains. the low-frequency image by filtering the inpiit image Xq by he in the 
horizontal and vertical directions, as follows: 

N N. 

^ Xi(i. J) = S S Xo(l * k. J * 1) hoCk) hod) 

k=l-N 1=1-N 

The digital image decoder is identical to the decoder in FIG. 2, except for the operation of the edge syn- 
50 thesizer 34 and inverse wavelet transform processor 36. The decoding description will be restricted to the op- 
eration of these elements. 

The second embodiment employs a high-pass filter ge with 2N taps, obtained from the smoothing filter he 
by the same relation as in the first embodiment: 

ge(k) = (-iyhe(-k) (k = -N...,N - 1) 
55 Whereas the filter go had even symmetry around zero, ge has odd symmetry around -1/2: 

ge(-k) = -ge(k - 1) (k = 1.2 N) 

As before, the Fourier transform Ge(G>) of ge is the complex conjugate of He(oi> n), and filters he and ge 
form a complementary pair: 
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|He(<D)|2 + |Ge(0>)|2 = 1 

The edge synthesizer 34 in the second embodiment filters the decoded edge images Sv and Sh by an 
edge synthesis filter obtained by filtering a standard edge with the high-pass filter ge. This process could be 
carried out directly, as in the first embodiment, but in that case the following computational problem would arise. 

5 When a uniform gradation as in FIG. 5 is filtered by g©, the result has a constant value at every point, but 

this value is not in general zero. Accordingly, the results of filtering the standard edges in FlGs. 3 and 4 with 
ge need not be the same. More significantly, their non-zero values will not necessarily be limited to the vicinity 
of the edge point, so direct filtering by an edge synthesis filter as in the first embodiment would require much 
computation. ^, 

10 If the differences between adjacent coefficient values in the edge synthesis filter are taken, however, these 

differences have the desirable properties of depending only on the edge sharpness, and being zero outside 
the vicinity of the edge point Accordingly, the edge synthesizer 34 in the second embodiment is preferably 
configured as in FIG. 6, comprising a horizontal differential edge synthesizer 44, vertical differential edge syn- 
thesizer 45, horizontal accumulator 46, and vertical accumulator 47. The horizontal and vertical differential 

15 edge synthesizers 44 and 45 employ a differential edge synthesis filter with coefficients de(t) obtained by fil- 
tering a standard edge of unit sharpness with the high-pass filter ge, then taking differences between the re- 
sulting values at adjacent pixels. Specifically, 

N-1 

20 de(t) = de(-t) = E ge(*«> if O < t < N 

k=t 



de(t) = de(-t) = 0 Ift^N 
25 . Whereas the high-pass and low-pass filters ge and he have an even number (2N) of taps, the differential 
edge synthesis filter de has an odd number (2N - 1) of taps. 

The horizontal differential edge synthesizer 44 filters the horizontal edge image Sh horizontally by de(t) 
to obtain a horizontal differential high-frequency image A Yh. The vertical differential edge synthesizer 45 fil- . 
ters the vertical edge image Sv vertically with de(t) to obtain a vertical differential high-frequency image A Yv. 
30 These differential high-frequency images are described by the following equations: 

N-1 

AYh(i, J) = Z Sh(i + t, j)-de{t) 



35 



40 



55 



t=l-N 



N-1 

AYv{i. j) = X Sv(l, j + t)de(t) 
t = l-N 



The horizontal and vertical accumulators 46 and 47 then complete the edge synthesis filtering operation 
and generate the horizontal and vertical high-frequency images Yh and Yv by performing accumulation oper^- 
ations on A Yh and A Yv in the horizontal and vertical directions, respectively. If the horizontal and vertical co- 
45 ofdinates i and j both start at zero, these accumulation operations can be described as follows: 

•1 

Yh(i, J) - S AYh(k. J) 
50 k=0 



J 

Yv(i, J) = £ AYv(l, k) 
k=0 

The digital image decoder also employs a completion filter fe defined by its Fourier transform Fe in the 
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same way as in the first embodiment: 

The inverse wavelet transform processor 36 in the second embodiment uses the conjugates f*e. g*©, and 
h*eOf filters fe, g©, and he, defined as explained earlier, e.g. he*(-k) = he(k). The inverse wavelet transform proc- 
essor 36 filters the decoded low-frequency image Xi' by h% horizontally and vertically, filters the horizontal 
high-frequency image Yh by g*e horizontally and vertically; filters the vertical high-frequency image Yy by 
g*e vertically and Te horizontally, and adds the three results to obtain the reconstructed image Xo', as follows: 

/ N-1 N-1 

Xo'(i, J) = Z S Xi*(i + k. J + 1) h*e(k) rh»e(l) 
k=-N 1=-N 

N N-1 

+ 15: Yh(i + k. j + I) 'e*Q{k) ' V*^{1) 
k=l-N k=-N 

N-1 N 

4- S S Yv(i > k. J + 1) -g*e(l) ■f*e(k) 
k=-N 1=1-N 



The inverse wavelet transfonrns employed in the first and second embodiments are essentially the same, 
because the even filters employed in the first embodiment are equal to their own conjugates. 
25 The difference being using filters with an odd number of taps as in the first embodiment and an even num- 

ber of taps as in the second emt>odlment is mainly computational. In terms of image encoding and reconstruct 
tion, both embodiments provide similar effects. 



Third embodiment 



The third embodiment is similar to the first embodiment, but the smoothing and inverse wavelet transform 
operations are cascaded over n wavelet scales, where n is an arbitrary integer greater than one, with edge 
detection and edge synthesis performed at each scale. 

Referring to FIG. 7, the digital image encoder in the third embodiment comprises n cascaded smoothing 
35 filters 20-1 to 20-n, n edge detectors 21-1 to 21-n, and a selector 50. It also comprises the same encoding 
section 22 and down sampler 23 as in the first embodiment in FIG. 1 , the encoding section 22 having the same 
encoders 24 and 25 and multiplexer 26. 

The image Xo input at the input terminal 10 is supplied to the first smoothing filter 20-1 and edge detector 
21-1 . Each smoothing filter 20-m in the cascade (m = 1 to n - 1 ) outputs a filtered image X^ to the next smooth- 
40 ing filter 20-(m + 1 ) and next edge detector 21 -(m + 1 ). The last smoothing filter 20-n outputs a low-frequency 
image X^ to the down sampler 23. 

The edge detectors 21-m are all identical to the edge detector 21 in FIG. 1. Each produces a pair of hori- 
zontal and vertical edge images Sh^ and Sv^, which are input to the selector 50. The selector 50 selects one 
of these pairs of edge images, and supplies the selected pair Sh and Sv to the edge image encoder 24. 
45 Referring to FIG. 8. the digital image decoder in the third embodiment comprises the same decoding sec- 

tion 28 and up sampler 35 as in the preceding embodiments, the decoding section 28 comprising the same 
demultiplexer 30 and decoders 31 and 32. The decoded low-frequency image output by the up sampler 35 is 
now denoted X„*. The digital image decoder also comprises a distributor 51, n edge synthesizers 34-m (m = 
1 to n), and an inverse wavelet transform processor 36. The inverse wavelet transform processor 36 now con- 
so sists of n cascaded inverse wavelet processors 36-m. 

The distributor 51 distributes identical copies of the horizontal and vertical edge images Sh and Sv received 
from the edge information decoder 31 to all of the edge synthesizers 34-m (m = 1 to n). Each edge synthesizer 
34-m outputs a pair of horizontal and vertical high-frequency images Yh^ and Yv^ to the corresponding inverse 
wavelet processor 36-m. Each inverse wavelet processor 36-m (m = 1 to n - 1) also receives a partially recon- 
55 structed image X^ from the preceding inverse wavelet processor 36-(m + 1 ) in the cascade, and outputs a 
partially reconstructed image X,n_ i to the next inverse wavelet processor 36-(m - 1). The first inverse wavelet 
processor 36-n in the cascade receives the decoded low-frequency image Xn* from the up sampler 35. The 
last inverse wavelet processor 36-1 outputs the fully reconstructed image Xq' to the input/output device 37. 
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Next the operation of the third embodiment will be described, starting with the encoding operation. 

Each of the smoothing filters 20-m in FIG. 7 employs a different set of filter coefficients hom(k)- All of these 
filters have properties similar to the filter ho employed in the first embodiment: even symmetry, and an odd 
number of taps with coefficients summing to plus or minus one. The first filter hoi satisfies the exact recon- 
struction condition. 

The cut-off frequency of each smoothing filter 20-m (m > 1) is one-half the cut-off frequency of the pre- 
ceding filter 20-(m - 1) In the cascade. Filters with these cut-off frequencies can be obtained by doubling the 
tap spacing from one filter to the next. That Is, hom can be obtained from ho(m- 1) as follows: 

hoJ2k) = ho(„i- i)(k) 
hom{2k + 1) = 0 (k = 0, ±1, ±2. ...) 

In smoothing filter 20-m, image X^. i is filtered horizontally and vertically by filter hom to obtain Image X^- 
As the image passes through the cascade, variations on progressively larger scales are smoothed out, and 
resolution of fine detail decreases correspondingly. 

Incidentally, If the low-pass filters hom (^^ ^ 1) are obtained by doubling the tap spacing as described atx)ve; 
they will pass high-frequency variations in certain regions above their cut-off frequencies. This is allowable 
because those high-frequency variations will already have been removed by previous filters In the cascade; 

Each edge detector 21 -m detects edges in the received image X^. i by the difference-of-differences meth- 
od described in the first embodiment. All of the edge detectors 21-m can use the same threshold T, or different 
thresholds can be employed in different edge detectors. 

In selecting one of the pairs of horizontal and vertical edge images Sh^ and Sv^ (m = 1 to n) produced by 
the edge detectors 21-m, the selector 50 may select according to statistics of the edge images themselves, 
such as the number of edge points detected, or according to an external command specifying a desired edge 
resolution. In general, edge images Shm and SVm obtained at lower resolutions (higher values of m) contain 
less information, but also contain less noise. 

The operations of down-sampling, encoding the selected edge images Sh and Sv, encoding the reduced 
image R, and multiplexing the encoded data are carried out as in the first embodiment 

Next the decoding operations will be described. These operations employ, edge synthesis filters iSom> com^ 
pletion filters fom* and high-pass filters Qom (m = 1 to n). Filters fom and gom are related to respective smoothing 
filters hom = 1 to n) In the same way that fo and go were related to ho in the first embodiment. Specifically, 

|Hom(<0)|2 + |Go,„(co)|2 =1 
rom(O)) 2 ; 

Once foi and goi have been obtained from hoi, the other filters fom and go^ = 2 to n) can be obtained 
by successively doubling the tap spacing, as was done for the low-pass filters hom, so that the number of taps 
remains 2N + 1 in each filter. 

Each edge synthesis filter eom is obtained by filtering a standard edge of unit sharpness with the corre- 
sponding high-pass filter gom- The length of each edge synthesis filter Oom (the distance between the twoout- 
enmost taps) is still less by two than the length of the corresponding high-pass filter gom» but eom niay have 
more taps than gom* because while gom nriay have many zero coefficients between the outermost two taps, eom 
in general will not. 

The operations of demultiplexing the encoded image C, decoding the edge information Cs, and decoding 
and up-sampling the low-frequency information Crare carried out as in the first embodiment, producing a pair 
of edge images Sh and Sv and a decoded low-frequency image Xn'. 

Each edge synthesizer 34-m (m = 1 to n) filters the pair of edge images Sh and Sv by its edge synthesis 
filter Oom to generate horizontal and vertical high-frequency images Yhm and Yvm. Even though all the edges 
were detected at a single selected resolution, the edge synthesizers 34-m synthesize high-frequency Images 
at different resolutions from these same edges. 

Each inverse wavelet processor 36-m generates the image Xm. i' from images X^. Yhm. and Yv^ in the 
same way that the inverse wavelet transform processor 36 in the first emt>odiment generated Xo from Xi. Yh, 
and Yv, That is, it filters Xm* horizontally and vertically by hom, filters Yhm horizontally by gom and vertically by 
fom filters Yvn, vertically by gom and horizontally by fom, and sums the three results. 

The compression ratios achieved by the third embodiment are comparable to those achieved by the first 
embodiment because the encoded information again comprises just one reduced low-frequency Image and 
two edge images, but the third embodiment gives better reconstruction of images with high-frequency noise. 
In the first embodiment, such noise can only be dealt with by raising the edge sharpness threshold T, which 
is not always a satisfactory solution. In the third embodiment, edges can be detected In an image from which 
high-frequency noise has been removed by smoothing, so it is not necessary to raise the edge sharpness 
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threshold. 

Fourth embodiment 

5 The fourth embodiment differs from the third embodiment in that the digital image encoder selects a par- 

ticular resolution before performing edge detection. The digital image decoder is the same as in the third enrv 
bbdiment 

Referring to FIG. 9. the digital image encoder has smoothing filters 20-m (m = 1 to n) that output filtered 
images as in the third embodiment. A selector 52 selects one of thesfe ^magesfor input to an edge detector 
10 21 , which is identical to the edge detector 21 in the first embodiment. Other elements in FIG. 9 are also identical 
to corresponding elements in the first and third embodiments, and have the same reference numerals. 

The selector 52 selects a particular image X„, according to statistics taken from the images (m = 1 to 
n), or according to an external command. The edge detector 21 detects edges in the selected image as in the 
first embodiment Other than this difference, the fourth embodiment operates like the third embodiment, so 
15 further description will be omitted. 

Thefourth embodiment has theadvantage ofasimpler configuration than the third embodiment, and need- 
less computation of edge images that wilt not be used is avoided. The selector 52 must, however, select a re- 
solution before seeing the results of edge detection. 

20 Fifth embodiment 

Thefifth embodiment differs from the third embodiment in using filters with an even number of taps. 
Referring to FIG. 10, the digital Image encoder of the fifth embodiment has the same structure as in the 
third embodiment, and the same reference numerals are used as in FIG. 7 except for the cascaded smoothing 
25 filters 54-m, which employ different filter functions hem (m = 1 to n). 

The filters hem are similar to the f ilter h© in the second embodiment in being loW-pass filters with even syrrv 
metry and an even number of taps. As in the third embodiment, each filter can be obtained from the pre- 
ceding filter by doubling the spacing between taps, which reduces the cut-off frequency by a factor of two. 
Other than the difference in the number of taps, the operation of the digital image encoders in the third 
30 and fifth embodiments is the same. 

Referring to FIG. 11, the digital image decoder in the fifth embodiment is similar to the digital image de- 
coder of the third embodiment (FIG. 8), except that the high-frequency images are generated by the method 
of FIG. 6. Elements that have the same reference numerals in FIGs. 11 and 8 are identical. 

The distributor 51 in the fifth embodiment distributes the pair of edge images Sh and Sv to pairs of dif- 
35 ferential edge synthesizers 56-m and 57-m (m = 1 to n)* which are similar to the differential edge synthesizers 
44 and 45 in FIG. 6. These output differential high-frequency images A Yhm and A Yvm to correspondlng pairs 
of accumulators 60-m and 61-m, which are similar to the accumulators 46 and 47 in FIG. 6. The accumulators 
.60-m and 61r-m output high-frequency images Yh„, and Yv^ to the inverse wavelet transform processor 36. 
The inverse wavelet transform processor 36 comprises inverse wavelet processors 62-n (m = 1 to n) which 
40 are similar to the inverse wavelet processors 36-n in the third embodiment^ except that they employ filters fem*, 
gem*, and hem* with an even number of taps. 

Filter h^* is conjugate to the low-pass filter hem employed in the encoder. Filters i^* and geh,* are related 
to hem* in the same way that f^ and go were related to h^ in the first embodiment Specifically, the Fourier trans- 
forms Fem*. Gem*, and Hem* of these filters satisfy: 

^ |Hem*(0>)|2 + |Ge„.*(<0)|2 = 1 

1 + |Hem*(a>)l^ 

2 

Filters fem* can be obtained from the above definitions of their Fourier transforms. Filters gem* can be con- 
structed as follows. 

so First, filters fei and gei are obtained from smoothing filter hei in the same way that fe and ge were obtained 

from he in the second embodiment: 

gei(k) = ( - 1)'^hei( - k) (k = - N .... N - 1) 
Each filter gem (m = 2 to n) is then obtained from the preceding filter g^^ - d by doubling the tap spacing. 
Each filter gem ("i = 1 to n) has odd symmetry. The order of taps of gem can be reversed to obtain the conjugate 
55 gen.*. 

The differential edge synthesizers 56-m and 57-m (m = 1 to n) employ respective differential edge syn- 
thesis filters dem that are obtained from the above high-pass filters gem in the same way that the filter de was 
obtained from ge in the second embodiment: by filtering a standard edge of unit sharpness with gem, then taking 
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the differences between resulting coefficients. . 

The decoding section 2B, up sampler 35, and distributor 51 operate as In the third embodiment The dif- 
ferential edge synthesizers 56-m and 57-m (m = 1 to n) operate like the differential edge synthesizers 44 and 
45 in the second embodiment, filtering the pair of edge images Sh and Sv by their respective differential edge 
5 synthesis filters de^. The accumulators 60-m and 61 -m operate like the accumulators 46 and 47 in FIG. 6. 

The inverse wavelet transform processor 36 operates as in the third embodiment. Each inverse wavelet 
processor 62-m generates Image X„>_iVfrom images X„,' , Yhm, and Yv^ by filtering Xm' horizontally and vert- 
ically with hem*, filtering Yh^ horizontally with gem* and vertically with fem*. filtering Yv^ vertically with gem* and 
horizontally with fem*. and summing the three results. ^, 
10 : The fifth embodiment has effects similar to the third embodiment. 

Sixth embodiment 

The sixth embodiment differs from the fifth as the fourth emt>odimeht differed from the third: the digital 
^5 Image encoder selects a particular resolution before performing edge detection. The digltalimage decoder |s 
?the sarhe as in the fifth embodiment 

- Referring to FIG. 12, the digital image encoder has the same smoothing filters 54-m (m = 1 to n) as in the 
fifth embodiment/which output filtered images Xm- As in the fourth embodiment, a selector 52 selects one of 
the Images X^ (m = 0 to n - 1). The other elements in FIG. 12 also operate as in the fourth embodiment, t>eing 
20 Identical to elements In 9 with the same reference numerals. 

Aside from the use of filters with an even number of taps, the sixth embodiment operates exactly like the 
fourth embodiment and provides the same effects, so further description will be omitted. 

Seventh embodiment 

25 

Theseventh embodiment is similartothe third embodiment, but is adapted to process moving Imageis com- 
prising successive frames. 

Referring to FIG. 13, the digitallmage encoder of the seventh embodiment has the same smoothing filters 
20-m and edge detectors 21-m (m = 1 to n) as the third embodiment, and the same down sampler 23 and se- 
30 lector 50. The encoding section 65, however, differs from the encoding section 22 in the third embodiment 

The encoding sectk3n 65 comprises a pair of motion estimators 66: and 67, a motion vector encoder 68, a 
pair of difference encoders 69 and 73, a pair of difference decoders 70 and 74, a pair of adders 71 and 75, 
and a pair of memories 72 and 76. 

Motion estimator 66 receives the pair of edge images Sh and Sv from selector 50, and corresponding pre- 
35 vious edge innages Sp from memory 72. The symbol Sp in the drawing denotes a pair of images: a previous 
horizontal edge Image and a previous vertical edge image. Motion estimator 66 outputs edge mottoh vectors 
Ms and a pair of horizontal and vertical edge difference images Dh and Dv. Motion estimator 67 receives the 
reduced inrrage Rfrom down sampler 23 and a previous reduced Image Rpfrom memory 76, and outputs low- 
frequency motion vectors Mr and a reduced difference image Dr. The motion vector encoder 68 encodes mo- 
^ tion vectors Ms and Mr and outputs motion information Cm. 

Difference encoder 69 encodes the edge difference images Dh and Dv and outputs edge difference in- 
formation Cds. This information is decoded by difference decoder 70 to produce a pair of decoded edge dif- 
ference images, which are denoted in the drawing by the single symbol D'. Similarly, difference encoder 70 
encodes the reduced difference image Dr and outputs low-frequency difference Information Cdr. Difference 
^ decoder 74 decodes Cdr and outputs a decoded reduced difference image Dr*. 

The multiplexer 78 multiplexes the edge difference information Cds, motion information Cm, and low-fre- 
quency difference information Cdr for output as an encoded image C. 

Referring to FIG. 1 4, the digital image decckier of the seventh embodiment is identical to the digital Image 
decoder of the third embodiment except for the decoding section 79. The decoding section 79 comprises a 
^ demultiplexer 80, a motion information decoder 82, a pair of difference decoders 81 and 83, a pair of motion 
compensators 84 and 86, and a pair of memories 85 and 87. 

The demultip)lexer 80 receives the encoded image C from the input/output device 27 and separates it into 
edge difference information Cds, motion information Cm, and low-frequency difference information Cdr. Dif- 
ference decoder 81 decodes Cds to obtain a pair of decoded edge difference Images Dh' and Dv*. Motion in- 
55 formation decoder 82 decodes Cm to obtain edge motion vectors Ms and low-frequency motion vectors Mr. 
Difference decoder 83 decodes Cdr to obtain a decoded reduced difference Image Df. 

Motion compensator 84 receives Dh\ Dv' and Ms from decoders 81 and 82, receives a pair of previous 
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edge images Sp from memory 85, and outputs a pair of edge images Sh and Sv to the distributor 51. Motion 
compensator 86 receives Mr and Dr' from decoders 82 and 83, receives a previous reduced image Rp' from 
memory 87» and outputs a decoded reduced image R' to the up sampler 35. 

Next the operation of the seventh embodiment will be described. The description will be confined to the 
5 encoding section 65 and decoding section 79, since the other parts operate as in the third embodiment. 

Referring again to FIG. 1 3, motion estimator 67 estimates motion in the reduced image R by. for example, 
the block-matching method, which involves computing the following quantity d(k, I) for blocks of pixels in the 
reduced innage R and previous reduced image Rp, using various vectors (k, 1)^ 

\ . - ■ ■ 

d(k, 1) = X IR(i, j) - Rp(i + k. J + 1)1 
1. J 

That is, motion estimator 67 finds the total difference in pixel values between a block in the current reduced 
15 image R and a tiiock displaced by k pixels horizontally and I pixels vertically from this block in the previous 
reduced image Rp. This computation is carried out for all vectors (k, I) within a designated search range; then 
the minimum resulting value d(k, I) is selected and its vector (k, I) is output as a motion vector. The entire pro- 
cedure is repeated for other blocks in the current reduced image, until motion has been estimated throughout 
the image, thereby obtaining the low-frequency motion vectors Mr. . 
20 The reduced difference image Dr consists of the differences in pbcel values between the current iand pre- 

vious reduced. images that cannot be accounted for by the detected motion. That is, it consists of values of 
the form 

Dr(i.j) = R(i.j) - Rp(i + k,j + I) 

where (k, I) is the motion vector of the block containing pixel R(i, j). 
25 Motion estimator 66 carries out this block-matching process separately on the horizontal aiid vertical edge 

images Sh and Sv. The motion vector encoder 68 encodes the motion vectors Ms and Mr by, for example, a 

variable-length encoding method. 

. Difference decoder 69 encodes the edge difference innages Dh and Dv by, for example, the same method 

as used by the edge image encoder 24 in the preceding embodiments. Difference encoder 74 encodes the 
30 reduced difference image by, for example, the same method as used by the low-frequency image encoder 25 

in the preceding embodiments. The difference decoders 70 and 74 employ methods reverse to those of the 

difference encoders 69 and 73 to obtain the decoded reduced difference image Dr' and the pair of decoded 

edge difference images D'. 

Adder 75 adds Dr' to the previous reduced image Rp and stores the resulting new reduced imagei in mem- 
35 ory 76 for use as a previous reduced image in the next frame. Similarly, adder 71 adds the pair of decoded . 

edge difference images D' to the previous pair of edge imaiges Sp and stores the resulting pair of new edge 

images in memory 72 for use as previous edge images in the next.frame. 

Memory 76 has space for storing at least two complete reduced images, the previous imajge read by motion 

estimator 67 and the new image stored by adder 75 being stored in different memory locations. In one storage 
40 scheme the roles of these locations are swapped at every frame, so what was the new image in one frame 

becomes the previous image in the next frame. Memory 72 can employ a similar scheme for storing edge inrv 

ages. 

Block matching requires a large number of arithmetic operations, particularly in motion estimator 66, which 
must estimate motion in two full-size images Sh and Sv. Motion estimator 66 is preferably adapted to reduce 

45 the amount of computation by restricting the search range. For example, motion in aframe can to some extent 
be predk:ted from motion in preceding frames, and the search range can be narrowed on the basis of these 
predictions, a technique well known in the prior art Also, after motion vectors for one of the two edge images 
Sh and Sv have been obtained, they can be used to narrow the search range for motion vectors in the other 
edge image, in the extreme case, block matching can be perfomried for just one of the two edge images^ and 

50 the resulting motion vectors applied to both edge images. 

Needless to say, the motion-vector search for a block terminates as soon as a vector (k, I) with d(k, I) = 0 
is found. This tends to happen quickly for many blocks in the edge images, because most pixel values in an 
edge image are usually zero. That is, edge detection in itself generally reduces the amount of motion-vector 
computation. 

55 Referring again to FIG. 14, to describe the operation of decoding section 79, after the demultiplexer 80 

separates the encoded image signal C into Cds, Cm, and Cr, the difference decoder 81 decodes Cds to ob- 
tained decoded edge difference images Dh' and Dv'. These images are klentical to the decoded edge differ- 
ence images obtained by the difference decoder 70 in the digital image encoder and denoted D' in FIG. 13. 
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The motion Information decoder 82 decodes Cm to obtain decoded motion vectors Ms and Mr. The motion vec- 
tor encoding scheme is preferably a lossless encoding scheme, so these vectors Ms and Mr are identical to 
the motion vectors Ms and Mr in FIG. 13. The difference decoder 83 decodes Cdrto obtain a decoded reduced 
image Dr' which is identical to the decoded reduced image Dr' obtained by difference decoder 74 in FIG. 13. 

5 For each block of difference values in the decoded reduced difference image Dr', motion compensator 

86 obtains the motion vector (k, I) of the block from the motion information decoder 82, reads a previous block 
displaced by (k, I) in the previous image Rp stored in memory 87, and adds the difference values to this previous 
block to obtain a block in the decoded reduced image R*. After obtaining the complete reduced image R\ it 
stores this newly decoded reduced image R' in memory 87 for use as thVprevious reduced image Rp in the 

10 next frame. Operating in a similar fashk>n, nrK>tion compensator 84 obtains horizontal and vertical edge images 
Sh and Sv from the decoded edge difference images Dh* and Dv\ motion vectors Ms, and previous edge im- 
ages Sp. and stores the new edge images Sh and Sv in memory 85 for use as previous edge images in the 
next frame. Memories 85 and 87 employ storage schemes similar to those of memories 72 and 76 in the digital 
image encoder. 

15 Due to quantization error, for example, the decoded difference images Dh\ Dv', and Dr' obtained by dif- 

ference decoders 81 and 83 in the digital image decoder will not necessarily be identical to the difference inv^ 
ages Dh, Dv, and Dr output by motion estimators 66 and 67 in the digital image encoder. They will, however, 
be identical to the decoded difference images obtained by difference decoders 70 and 74 in the digital image 
encoder, as noted above, so the new images stored in memories 85 and 87 in the digital image decoder will 

20 be identical to the new images stored in memories 72 and 76 in the digital image encoder. 

Accordingly, the motion estimators 66 and 67 and motion compensators 84 and 86 wj II always estimate and 
compensate for motion from the same previous images, even if the difference images are not decoded with 
perfect accuracy. 

The combination of edge detection, down-sampling, and motion estimation and compensation enables 
25 very high video compression ratios to be achieved, particularly for Images that tend to remain largely station- 
ary. Such images Include the Images commonly encountered in video teleconferencing, video games, and a 
wkie variety of so-called multimedia systems, which provide a video interface between a person and a conv 
puter. The present invention can be practiced to good advantage in systems like these. 

30 Eighth embodiment 

The eighth embodiment combines features of the fourth and seventh emfc>odiments. 

Referring to FIG. 15, the digital image encoder of the eighth embodiment has the same smoothing filters 
20-1 to 20-n, edge detector 21, down sampler 23, and selector 52 as the fourth embodiment (FIG. 9), and the 
35 same motion-estimating encoding section 65 as the seventh embodiment (FIG. 1 3). The digital image decoder 
of the eighth embodiment is identical to the digital image decoder of the seventh embodiment (FIG. 14). 

The eighth embodiment operates as explained in the fourth and seventh embodiments, so further descrip- 
tion will be omitted. By reducing the amount of edge detection, the eighth embodiment speeds up the encoding 
of moving images and enables frames to be encoded at a faster rate than the seventh embodiment 

40 

Ninth embodiment 

The ninth embodiment is similar to the eighth embodiment, with further provisions for speeding up the com- 
putation of motion vectors. 

45 Referring to FIG. 16, the encoding section 89 of the digital image encoder of the ninth embodiment is similar 

to the encoding section 65 in FiGs. 13 and 15, but has an additional bidirectional interconnection 90 between 
motion estimators 66 and 67. This enables motion estimator 67 to receive the edge motion vectors Ms found 
by motion estimator 66, scale them down to the size of the reduced image R, and restrict the search for motion 
vectors in the reduced image R to vectors close to the scaled-down edge motion vectors. In the extreme.case, 

50 the search can be eliminated entirely: the low-frequency motion vectors Mr can k>e computed directly from the 
edge motion vectors Ms. 

Alternatively, motion estimator 66 can receive the motion vectors Mr found by block matching In the re- 
duced image R, scale them up to the size of the edge images Sh and Sv, and restrict the search for motion 
vectors for Sh and Sv to vectors close to the scaled-up vectors. This scheme is particularly effective in detecting 
55 sudden, large motion. 

Aside from this difference, the ninth embodiment has the same structure as the eighth embodiment and 
operates in the same way, so further description will be omitted. By narrowing down the search for motton vec^ 
tors, the ninth embodiment enables frames to be encoded at a still faster rate than the eighth embodiment 

15 
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Tenth embodiment 

The tenth embodiment combines features of the fifth and seventh embodiments. 
Referring to FIG. 17, the digital image encoder of the tenth embodiment has the same smoothing filters 
5 54-1 to 54-n, edge detectors 21-1 to 21-n, down sampler 23, and selector 50 as the fifth embodlrnent (FIG. 
10), and the same encoding section 65 as the seventh embodiment (FIG. 13). Referring to FIG. 18, the digital 
image decoder of the tenth embodiment has the same decoding section 79 as the seventh iembodlment (FIG. 
14). and the same up sampler 35, distributor 51, differential edge synthesizers 56-1 to 57-n, accumulators 60- 
1 to 61-n, and inverse wavelet transform processor 36 as the fifth embodiment (FIG; 11). The tenth embodi- 
10 ment operates as explained in the fifth and seventh embodiments, and provides similar effects, so further 
description will be omitted. 

Eleventh embodiment 

15 The eleventh enibodiment combines features of the sixth and seventh embodiments. 

Referring to FIG. 19, the digitalimage encoder of the eleventh embodiment has the same smoothing filters 
54t1 to 54-n, edge detector 21 , down sampler 23. and selector 52 as the sixth emt>odiment (FIG. 12). and the 
same encoding section 65 as the seventh embodiment (FIG. 13). The digital inriage decoder of the eleventh 
embodiment is identical to the digital image decoder of the seventh embodiment (FIG. 14). The eleventh em- 

20 bodiment operates as explained in the sixth and seventh embodiments, and provides similar effects, so further 
description will be omitted. 

■ Twelfth embodiment 

25 The twelfth embodiment combines features of the ninth and eleventh embodiments. 

Referring to FIG. 20, the digital image encoder of the twelfth embodiment has the same smoothing filters 
54-1 to 54-n, edge detector 21 , down sampler 23^ and selector 52 as the eleventh embodiment (FIG. 18), and 
the same encoding section 89 as in the ninth embodiment (FIG. 16), with a bidirectional interconnection 90 
between the motion estimators 66 and 67. The digital image decoder of the eleventh embodiment Is identical 

30 to the digital image decoder of the tenth and eleventh embodiments (FiG. 18). The twelfth embodiment oper- 
ates as explained in these preceding embodiments, and provides similar effects, so further description will 
be omitted. 

As described above, the invented encoding and decoding method and devices provide high compression 
rates with modest amounts of computation. Although there is some loss of detail, reconstructed images are 
35 not distorted, and because high-frequency information is synthesized in a single step instead of by iteration, 
the reconstruction process is reliably completed within a short and definite time. The invention can be usefully 
applied to both still and moving images for a wide variety of purposes, in a wide variety of systems. 

The scope of the invention is not restricted to the embodiments described above, but includes such further 
variations as the following. 

40 Instead of producing separate horizontal and vertical edge images, the edge detector or detectors ban out- 

put a single edge image in which each edge point has lx)th a sharpness value S(i, j) and direction angle 0 (i, 
j), computed from Sh(i, j) and Sv(i, j) by, for example, the following formulas: 

S(l,j) = sgnISh(i,j)HSh(i,j)2 + Sv(i,j)2]i/2 

e(i.j) = arctanISv(i. j)/Sh(i, j)] 
45 In these fornnulas. sgn denotes the alget)raic sign (plus or minus one), and arctan the inverse tangentf unc- 

tion. The encoding section then has only one set of edge positions to encode. The decoding section in the digital 
image decoder can recover Sh(i, j) and Sv(i, j) from S(i, j) and 9 (i, j). 

This method of encoding only a single edge image is particularly effective if chain encoding is employed, 
because the direction angle tends to change only slowly along a chain of edge points. Moreover, the angle 
50 tends to be perpendicular to the direction of the chain, so it is possible to have the edge detector calculate only 
the above edge sharpness S(i, j), and let the decoder infer the direction angle 9 (i, j) of an edge point (i, j) from 
the direction of the chain in the vicinity of that point. The compression ratio can be significantly improved in 
this way, although with some loss of reconstruction accuracy. 

If the seventh to twelfth embodiments are varied in this way, the new and previous edge images Sp stored 
55 in memories 72 and 85 can be a single edge image with pixel values S(i, j). After storing a single new edge 
image in memory 85, motion compensator 84 infers 0 (i, j), calculates Sh(l, j) and Sv(i, j), and generates hor- 
izontal and vertical edge images Sh and Sv for output to the distributor 51 . 

The number of smoothing filters need not be the same as the number of edge synthesizers. If the desired 
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edge resolution is known in advance, the digital image encoder requires no more than two smoothing filters: 
one to smooth the input image to the desired resolution for edge detection,, and another to obtain the low-fre- 
quency image by further smoothing. Despite this, the digital image decoder may employ an arbitrary number 
of edge synthesizers for reconstruction. In this case it is not necessary for the low-pass filters in the digital 
5 image encoder and high-pass filters in the digital image decoder to form complementary pairs. 

Obtaining a series of low-pass or high-pass filters by successive doubling of the tap spacing is computa- 
tionally convenient, but the low-pass and high-pass filters do not have to be obtained in this way, and their 
cut-off frequencies to not have to be related by factors of one-half. Moreover, the low-pass filters in the digital 
image encoder do not have to be cascaded; a set of parallel low-pass filters could be used, 
fo Since the difference-of-differences edge detection method is equivalent to filtering by a second-derivative 

filler d with coefficients (1, -2, 1), the encoder could also be structured so that the input image is filtered by 
a cascaded series of filters comprising d, h^, hz, hn. i to detect edge points at various resolutions. The input 
image can then be separately filtered by a single filter equal to the convolution of h^, h2, hn to obtain the 
low^frequency image (hi, hn are low-pass filters). 
f5 Edge detection is not limited to the difference-of-differences method. This method is preferred, but othisr 

methods can be employed. 

The standard edge from which the edge synthesis fitters are generated need not be an edge of the type 
shown in FIGs. 3 and 4, in which pixel values change at first one constant rate, then another constant rate. 
Other types can be used,4)rovided the standard edge has only one point which would be detected as an edge 
20 point by the edge detector. 

Down-sampling and up-sampling can be omitted, if the encoding method employed for the low-frequency 
image provides adequate data compression without down-sampling. 

In the seventh to twelfth embodiments. If the difference images Dh, Dv, and Dr produced by motion es- 
. timation are encoded by a lossless encoding scheme, nriemories 72 and 76 in the digital image encoders can 
25 be updated by direct input of edge images Sh and Sv and reduced Image R. Difference decoders 70 and 74 
and adders 71 and 75 can then be omitted. 

Motion estimation and compensation need not be performed in every frame. These processes can be omit- 
ted occasionally, to prevent unchecked propagation of errors from frame to frame. 

Motion estimation and compensaition can of course be added to the one-stage digital ! mage encoders and 
30 decoders of the first two embodiments^ to compress moving images. 

Those skilled in the art will recognize that still further variations are possible without departing from the 
scope claimed below. 



35 Claims 

1. A method of encoding and decoding a digitized image consisting of pixel values, comprising the stisps of: 
detecting sharpness of edges in said digitized image; 

encoding position and sharpness values of edge points having sharpness values exceeding a cer- 
- tain threshold, thereby generating edge Information; 

filtering said digitized image by a low-pass filter, thereby generating a low-frequency Image; 
encoding said low-frequency image, thereby generating low-frequency information; 
sending said edge information and said low-frequency information to an input/output device (27); 
receiving said edge information and said low-frequency information from sakl input/output device 

(27); 

generating a horizontal edge Image and a vertical edge Image from sakJ edge Information; 
synthesizing a pair of high-frequency images by filtering said horizontal edge Image and said vert- 
ical edge image with an edge synthesis filter, 

decoding said tow-frequency information, thereby obtaining a decoded low-frequency image; and 
performing an inverse wavelet transform on said decoded low-frequency image and said pair of 
high-frequency images. 

2. The method of claim 1 , wherein detecting sharpness of edges comprises: 

finding differences between said pixel values; and 
55 taking differences of said differences. 

3. The method of claim 1 , wherein encoding said low-frequency image comprises down-sampling.. 
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4. The method of claim 1 , wherein said digitized image comprises successive frames, and encoding position 
. and sharpness values comprises motion estimation. 

5. The method of claim 1 , wherein said low-pass filter has even symmetry. 

6. The method of claim 1, wherein filtering said digitized image comprises filtering by a series of low-pass 
filters having different cut-off frequencies, thereby generating a series of images with different resolu- 
tions, including at least said digitized image as a highest-resolution image and said low-frequency image 
as a lowest-resolution image. \. ' 

7. The method of claim 6, wherein detecting sharpness of edges comprises: 
detecting sharpness of edges in at least two images among said series of images, excluding said 

low-frequency image; and 

selecting one of said at least two images for encoding of said position and sharpness values. 

8. The method of claim 6. wherein detecting sharpness of edges comprises: 

selecting one image among said series of images; and 
detecting sharpness of edges in the image thus selected. V 

20 9. The method of claim 1 .wherein said edge synthesis filter has coefficients that ai^e generated by filtering 
a standard edge by a high^pass filter. 

10. The method of claim 9, wherein said standard edge consists of a first part and a second part, with pixel 
values changing at one constant rate in sald first part and at a different constant rate in said siecond part 

25 . 

11. The method of claim 9, wherein: 

said step of synthesizing comprises filtering said horizontal edge image with at least two edge syn- 
thesis filters and filtering said vertical edge image with said edge synthesis filters, thereby synthesizing 
. at least two pairs of high-frequency images; 
30 said inverse wavelet transform is performed on all said pairs of high-frequency images thus syn- 

thesized; and 

said edge synthesis filters are obtained by filtering said standard edge with high^pass filters having 
different cut-off frequencies. 

35 12. The method of claim 9, wherein said high-pass filter has even symmetry. 

13. The method of claim 9, wherein said high-pass filter has odd symmetry. 

14. The method of claim 1 3; wherein said step of synthesizing comprises: 

40 taking differences between consecutive coefficients of said edge synthesis filter, thereby gener- 

ating a differential edge synthesis filter; 

filtering said horizontal edge image and said vertical edge image by said differential edge synthesis 
filter, thereby generating a pair of differential high-frequency images; and 

accuhnulating values in said differential high-frequency images. 

45 

15. A method of encoding a digitized image consisting of pixels having pixel values; comprising the steps of: 

filtering said digitized irhage by a low-pass filter, thereby generating a low-frequency image; 
down-sampling said low-frequency image, thereby generating a reduced image smaller in size than 
said digitized image; 

50 . encoding said reduced image, thereby generating fow-frequency information; 

generating an edge image by detecting edges In said digitized image; 
encoding said edge image, thereby generating edge information; and 

sending said edge information and said low-frequency information to an input/output device (27). 
55 1 6. The method of claim 1 5, wherein said low-pass filter has even symmetry. 

1 7. The method of claim 1 5, wherein said low-pass filter satisfies an exact reconstruction condition. 
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1 8. The method of claim 1 5, wherein detecting edges comprises detecting variations in rate of change of said 
pixel values from pixel to pixel. 

19. The method of claim 1 5. wherein generating an edge image comprises: 

5 detecting edges in a horizontal direction, thereby generating a horizontal edge image; and 

detecting edges in a vertical direction, thereby generating a vertical edge image. 

. . 20. The method of claim 1 5, said method being applied to successive frames of a moving image* wherein the 
step of encoding said edge image comprises: \ 
10 reading a previous edge image, pertaining to a previous frame, from a first memory (72); 

generating edge motion vectors by detecting motion of said edge image with respect to said pre- 
' vious edge image; 

encoding said edge motion vectors, thereby generating edge motion information as part of said 
edge information; 

15 generating an edge difference image by detecting differences between said edge image and said 

previous edge image, over and above said motion of said edge image; and 

encoding said edge difference image, thereby generating edge difference information as another 
part of said edge information. 

20 21. The method of claim 20, wherein encoding said edge image comprises the further steps of: 

decoding said edge difference information, thereby generating a decoded edge difference image; 
adding said decoded edge difference Image to said previous edge image to generate a new edge 
image; and 

storing .said new edge image in siaid first memory (72), for use as a previous edge image in a sub- 
25 sequent frame. 

22. . The method of claim 20, wherein the step of encoding said reduced image comprises: 

reading a previous reduced Image, pertaining to said previous frame, from a second memory (76); 
generating low-frequency motion vectors by detecting motion of said reduced image with respect 
30 to said previous reduced image; 

encoding said low-frequency motion vectors, thereby generating low-frequency motion information 
as part of said low-frequency information; 

generating a reduced difference image by detecting differences between said reduced Image and 
said previous reduced image, over and above said motion of said reduced Image; and 
35 encoding said reduced difference image, thereby generating low-frequency difference information 

as another part of said low-frequency information. 

23. The method of claim 22, wherein encoding said reduced image comprises the further steps of: 

decoding said low-frequency difference information, thereby generating a decoded reduced dif- 
40 ference image; 

adding said decoded reduced difference image to said previous reduced image to generate a new 
reduced Image; and 

storing said new reduced image in said second memory (76), for use as a previous reduced image 
in a subsequent frame. 
45 . 

24. The. method of claim 22^ wherein generating edge nrK)tion vectors comprises using said low-frequency 
motion vectors to select a range of search for said edge motion vectors. 

25. The method of claim 22, wherein generating iow-f requency motion vectors comprises using said edge 
50 , . motion vectors to select a range of search for said low-frequency motion vectors. 

26. A method of reconstructing a digitized image from encoded edge information and low-frequency infor- 
mation, comprising the steps of: 

generating a horizontal edge image and a vertical edge image by decoding said edge information; 
55 synthesizing a pair of high-frequency Images by filtering said horizontal edge image horizontally 

with an edge synthesis filter, and filtering said vertical edge image vertically with said edge synthesis filter; 
generating a reduced image by decoding said low-frequency information; 
uF>-sampling said reduced image, thereby generating a low-frequency image equal in size to said 
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high-frequency images; 

performing an inverse wavelet transform on said high-frequency images and said low-frequency 
image, thereby obtaining said digitized image; and 

sending said digitized image to an input/output device (37). 

5 

27. The method of claim 26, wherein said edge synthesis filter has coefficients that are generated by filtering 
a standard edge, having just one sharp variation, by a high-pass filter. 

28. The nnethod of claim 27, wherein said standard edge consists of a first part and a second part, with pixel 
10 values changing at one constant rate in said first part and at a different constant rate in said second part. 

29. The method of claim 27, wherein the step of performing an inverse wavelet transform comprises filtering 
said high-frequency Images by a conjugate fitter of said high-pass filter. 

IS 30. The method of claim 27, wherein said high-pass filter satisfies an exact reconstruction condition. 

31. The method of claim 27, wherein said high-pass filter has even symmetry. . 

32. The method of claim 27, wherein said high-pass filter has odd symmetry. 

20 

33. The method of claim 32, wherein filtering with said edgesynthesisf liter is performed by using a differential 
edge synthesis filter having coefficients equal to differences between consecutive coefficients of said 
edge synthesis filter, and said step of synthesizing a pair of high-frequency Images comprises: . 

filtering said horizontal edge image horizontally by said differential edge synthesis filter, thereby 
25 synthesizing a horizontal differential high-frequency image; 

accumulating values in said horizontal differential high-frequency image; 

filtering said vertical edge image vertically by said differential edge synthesis filter, thereby syn- 
thesizing a vertical differential high-frequency Image; and 

accumulating values In said vertical differential high-frequency image.* 

30 

34. The method of claim 26, said method being applied to successive frames of ian encoded moving image, 
wherein said edge information comprises edge difference Information and edge motion information, and 
decoding said edge information comprises: 

decoding said edge difference information to obtain an edge difference inr^ge; 
35 decoding said edge motion information to obtain edge motion vectors; 

reading a previous edge Image, pertaining to a previous frame, from a third memory (85); 

adding said edge difference image to said previous edge image according to said edge motion vec- 
tors, to obtain a new edge image; and 

storing said new edge image in said third memory (85) for use as a previous edge imiage of a sub- 
40 sequent frame. 

35. The method of claim 34, wherein said low-frequency information comprises low-frequency difference in- 
formation and low-frequency motion information, and decoding said low-frequency information compris- 
es: 

45 decoding said tow-frequency difference information to obtain a reduced difference image; 

decoding said low-frequency motion information to obtain low-frequency motion vectors: 
reading a previous reduced image, pertaining to said previous frame, from a fourth memory (87); 
adding said reduced difference image to said previous reduced jmage according to said low-fre- 
quency motion vectors, to obtain said reduced image; and 
so storing said reduced image In said fourth memory (87) for use as a previous reduced image of a 

subsequent frame. 

36. A method of encoding a digitized image consisting of pixels having pixel values, comprising the steps of: 

filtering said digitized image by a series of low-pass filters having progressively lower cut-off fre- 
ss quencies, thereby generating a series of filtered images culminating in a low-frequency ihfiage, which was 

filtered with a lowest cut-off frequency among said cut-off frequencies; 

down-sampling said low-frequency image, thereby generating a reduced image smaller in size than 
said digitized image; 
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encoding said reduced image, thereby generating low-frequency information; 
generating an edge image by detecting edges in one Image among said digitized Image and said 
filtered images, excluding said low-frequency image; 

" encoding said edge image, thereby generating edge information; and 
5 sending said edge information and said low-frequency information to an Input/output device (27). 

37. The method of claim 36, wherein said low-pass filters have even symmetry. 

c38. The method of claim 36, wherein detecting edges comprises detectihg variations in rate of change of in- 
10 tensity from pixel to pixel. 

39. The method of claim 36, wherein generating an edge image comprises: 

detecting edges in a horizontal direction, thereby generating a horizontal edge image; and 
detecting edges in a vertical direction, thereby generating a vertical edge image. 

40- The method of claim 36, wherein generating an edge image comprises: 

generating a plurality of edge images by detecting edges in a plurality of images among said digi- 
tized image and said filtered images, excluding said low-frequency image; and 
X selecting one of said plurality of edge images. 

20 

41- The method of claim 36, wherein generating an edge image comprises: 

selecting one image among said digitized image and said filtered images, excluding said low-fre- 
quency image; and 

detecting edges In said one image. 

25 . 

42. The method of claim 36, said method being applied to successive frames of a moving image, wherein the 
step of encoding said edge image comprises: 

reading a previous edge image, pertaining to a previous frame, from a first memory (72); 
generating edge motion vectors by detecting motion of said edge image with respect to said pre- 
30 vious edge image; 

encoding said edge motion vectors, thereby generating edge motion information as part of said 
edge information; 

. generating an edge difference image by detecting differences between said edge image and said 
previous edge image, over and above said motion of said edge Image; and 
35 encoding said edge difference image, thereby generating edge difference information as another 

part of said edge Information. 

43; The method of claim 42, wherein encoding said edge image comprises the further steps of: 
V ff, - decoding said edge difference information, thereby generating a decoded edge difference imagef; 

^ adding said decoded edge difference image to said previous edge image to generate a new edge 

7 / image- and 

storing said new edge image in said first memory (72), for use as a previous edge image in a sub- 
r . sequent frame. 

45 44. The method of claim 42, wherein, the step of encoding said reduced image comprises: 

reading a previous reduced Image, pertaining to said previous frame, from a second memory (76); 
generating low-frequency motion vectors by detecting motion of said reduced image with respect 
to said previous reduced image; 

encoding said low-frequency motion vectors, thereby generating low-frequency motion infonmatlon 
50 as part of said low-frequency Information; 

generating a reduced difference image by detecting differences between said reduced image and 
said previous reduced image, over and above said motion of said reduced image; and 

encoding said reduced difference image, thereby generating low-frequency difference information 
as another part of said low-frequency Information. 



55 



45. The method of claim 44, wherein encoding said reduced image comprises the further steps of: 

decoding said low-frequency difference information, thereby generating a decoded reduced dif- 
ference image; 
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adding said decoded reduced difference image to said previous reduced image to generate a new 
reduced image; and 

storing said new reduced Image in said second memory (76), for use as a previous reduced image 
in a subsequent f ranne. 

5 

46. The method of claim 44. wherein generating edge motion vectors comprises using said low-frequency 
motion vectors to select a range of search for said edge nrK)tton vectors. 

47. The method of claim 44, wherein generating low-frequency motion, vectors comprises using said edge 
10 motjon vectors to select a range of search for said low-frequency motion vectors. 

48. A method of reconstructing a digitized image from encoded edge information and low-frequency infor- 
mation, comprising the steps of: 

generating a horizontal edge image and a vertical edge image by decoding said edge information; 
15 synthesizing a series of pairs of high-frequency Images by filtering said horizontal edge image hor^- 

izontally with a series of edge synthesis filters^ and filtering said vertical edge Image vertically with sdid 

series of edge synthesis filters- 
generating a reduced image by decoding said low-frequency information; 
up-sampling said reduced image, thereby generating a low-frequency image equal in size to said 
20 high-frequency images; 

perfonming an Inverse wavelet transform on said series of pairs of high-frequency images and safid 

low-frequency image, thereby obtaining said digitized image; and 
sending said digitized Image to an input/output device (37). 

25 49. The method of claim 48, wherein said series of edge synthesis filters Is generated by filtering a standard 
edge, having just one sharp variation, by a corresponding series of high-pass filters with progressively 
decreasing cut-off frequencies. 

50. The method of claim 49, wherein said standard edge consists of a first part and a second part, with pixel 
30 values changing at one constant rate in said first part and at a different constant rate in said second part. 

51. The method of claim 49, wherein the step of performing an inverse wavelet transform comprises filtering 
said pairs of high-frequency images by respective conjugate filters of said series of high-pass filters. 

35 52. The. method of claim 49, wherein said high-pass fitters have even symmetry. 

53. The method of claim 49, wherein said high-pass filters have odd symmetry. 

54. The method of daim 53, wherein filtering said pair of edge Images by said edge synthesis filters com- 
40 prises: . 

taking differences between consecutive coefficients of said edge synthesis filters, thereby gen- 
. erating a series of differential edge synthesis filters; 

filtering said horizontal edge image horizontally by said differential edge synthes^ filters, thereby 
synthesizing a series of differential high-frequency Images; 
45 accumulating values in respective horizontal differential high-frequency images; 

filtering said vertical edge Image vertically by said differential edge synthesis filters, thereby syn- 
thesizing a series of differential high-frequency Images; and 

accumulating values in respective vertical differential high-frequency images. 

so 55. The method of claim 48, said method bieing applied to successive frames of an encoded moving image, 
wherein said edge information comprises edge difference information and edge motion information, and 
decoding said edge information comprises: 

decoding said edge difference information to obtain an edge difference image; 
decoding said edge motion Information to obtain edge motion vectors; 
S5 reading a previous edge image, pertaining to a previous frame, from a third memory (85); 

adding said edge difference image to said previous edge image according to said edge motion vec- 
tors,.to obtain a new edge image; and 

storing said new edge Image In said third memory (85) for use as a previous edge Image of a sub- 
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sequent frame. 

56. The method of claim 55, wherein said low-frequency information comprises low-frequency difference in- 
formation and low-frequency motion information, and decoding said low-frequency information compris- 

5 es: 

decoding said low-frequency difference information to obtain a reduced difference image; 
decoding said low-frequency motion information to obtain low-frequency motbn vectors; 
reading a previous reduced image, pertaining to said previous frame, from a fourth memory (87); 
. adding said reduced difference image to said previous redubed image according to said low-fre- 
10 quency motion vectors, to obtain said reduced image; and 

storing said reduced image in said fourth memory (87) for use as a previous reduced image of a 
subsequent frame. 

57. Adigital image encoder for encoding a digitized image consisting of pixels having pixel values, comprising: 
15 an input terminal (10) for receiving said digitized image; 

a smoothing fitter (20) coupled to said input terminal (10), for filtering said digitized image to gen- 
erate a low-frequency image; 

a down sampler (23) coupled to said snrK)othing filter (20), for down-sampling said low-frequency 
... image to generate a reduced image; 
20 an edge detector (21) coupled to said input terminal (1 0), for generating an edge image by detecting 

edge points in said digitized image; and 

an encoding section (22) coupled to said edge detector (21) and said down sampler (23)^ for en- 
coding said edge image and said reduced Image. 

25 58. The digital image encoder of claim 57, wherein said edge detector (21 ) detects variations in rate of change 
of said pixel values from pixel to pixel. 

59. The digital image encoder of claim 57, wherein said smoothing filter (20) is a low-pass filter with even 
symmetry. 
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60. The digital image encoder of daim 58, wherein said low-pass filter satisfies an exact reconstruction con- 
dition. 



61. Adigital image encoder for encoding a digitized image consisting of pixels having pixel values, comprising: 
35 an input terminal (10) for receiving said digitized image; 

a series of smoothing filters (20-m) coupled to said input terminal (10), having progressively lower 
cut-off frequencies, for filtering said digitized image to generate a series of filtered images culminating 
in a low-frequency Image, which was filtered with a lowest cut-off frequency among said cut-off frequen- 
cies; 

40 a down sampler (23) coupled to a last one of said smoothing filters (20-n) in said series^f^d^n^^^ '^^^^ 

sampling said low-frequency image to generate a reduced image; i-;. .- .. 

a plurality of edge detectors (21 -m) for generating respective edge images by detectlng gS^^^^^^ffg^ * % 

in a plurality of images among said digitized image and said filtered images, exduding said low-freiqiienc^^^^^*^^^^ 
image; 

45 aselector(50)coupledtosaidpluralityof edge detectors (21 -m), for selecting an edge infiage from 

among said edge images; and 

an encoding section coupled to said selector (50) and said down sampler (23), for encoding said 
edge image and said reduced image. 

50 62.. The digital image encoder of claim 61, wherein said smoothing filters (20-m) are coupled in cascade to 
said input terminal (10). 

63. The digital image encoder of claim 61, wherein said smoothing filters (20-m) are low-pass filters with even 
symmetry. 



64. The digital image encoder of claim 61, wherein said edge detectors (21-m) detect variations in rate of 
change of said pixel values from pixel to pixel. 
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65. The digital image encoder of claim 61, wherein said encoding section (22) comprises: 

an edge image encoder (24) coupled to said selector (50), forencoding said edge image to generate 
edge information; 

a low-frequency image encoder (25) coupled to said down sampler (23), for encoding said reduced 
5 image to generate low-frequency information; and 

a multiplexer (26), coupled to said edge image encoder (24) and said low-frequency image encoder 
(25), for multiplexing said tow-frequency information and said edge information. 

66^ The digital image encoder of claim 61, wherein said digitized imageN^ a moving image consisting of sue- 
10 cessive frames, and said encoding section (65) comprises: 

a first memory (72) for storing a previous edge image; pertaining to a previous frarhe among said 
frames; 

a first motion estimator (66) coupled to said selector (50) and said first memory (72), for detecting 
motion of said edge image with respect to said previous edge image, generating edge motion vectors, 
15 detecting differences between said edge image and said previous edge image over and above said mo- 

tion of said edge image, and generating an edge difference image; 

a second memory (76) for storing a previous reduced image, pertaining to said previous frame; 

- • .and 

a second motion estimator (67) coupled to said down sampler (23) and said second mennory (76). 
20 for detecting motion of said reduced image with respect to said previous reduced image, generating low- 

frequency motion vectors, detecting differences between said reduced image and said previous reduced 
image over and above said motion of said reduced image, and generating a reduced difference image. 

67. The digital image encoder of daim 66, wherein: 

25 said first motion estimator (66) is coupled to said second nriotion estimator (67); and 

said first motion estimator (66) uses said low-frequency motion vectors to select a range of search 
for Siaid edge motion vectors. 

68. The digital image encoder of claim 66, wherein: 

30 said first motion estimator (66) is coupled to said second motion estimator (67); and 

said second motion estimator (67) uses said edge motion vectors to select a range of search for 
said low-frequency motion vectors. 

69. The digital image encoder of claim 66, wherein said encoding section (65) also comprises: 

35 a first difference encoder (69) coupled to said first motion estimator (66), for encoding said edge 

difference image, thereby generating edge difference information; 

a first difference decoder (70) coupled to said first difference encoder (69), for decoding said edge 
difference information, thereby generating a decoded edge difference image; 

a first adder (71) for adding said decoded edge difference inr^ge to said previous edge image to 
40 generate a new edge image, and storing said new edge image in said first memory (72) for use as a pre- 

vious edge image in a sut>sequent frame among said frames; 

a second difference encoder (73) coupled to said second motion estimator (67), for encoding said 
reduced difference image, thereby generating low-frequency difference information; 

a second difference decoder (74) coupled to said second difference encoder (73), for decoding 
45 said low-rfrequency difference information, thereby generating a decoded reduced difference image; 

a second adder (75) for adding said decoded reduced difference image to said previous reduced 
image to generate a new reduced difference image, and storing said new reduced difference image in 
said second memory (76) for use as a previous reduced image in said subsequent frame; 

a motion vector encoder (68) coupled to said first motbn estimator (66) and said second motion 
50 estimator (67), for encoding said edge motion vectors and said low-^f requency motion vectors to generate 

motion information; and 

a multiplexer (78) coupled to said motion vector encoder (68), said first difference encoder (69), 
and said second difference encoder (73), for multiplexing said motion information, said edge difference 
information, and said low-frequency information. 
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70. A digital image encoderfor encoding a digitized image consisting of pixels having pixel values, comprising: 
an input terminal (10) for receiving said digitized image; 

a series of smoothing filters (20-m) coupled to said input tenrninal (10), having progressively lower 
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cut-off frequencies, for filtering said digitized image to generate a series of filtered images culminating 
in a low-frequency image, which was filtered with a lowest cut-off frequency among said cut-off frequen- 
cies; 

a down sampler (23) coupled to a last one of said smoothing filters (20-n) in said series, for down- 
sampling said low-frequency image to generate a reduced image; 

a selector (52) coupled to said series of smoothing filters (20-m), for selecting an image from among 
said digitized image and said filtered images, excluding said low-frequency image; 

an edge detector (21 ) coupled to said selector (52), for generating an edge image by detecting edge 
points in the image selected by said selector (52); and \ 

an encoding section coupled to said down sampler (23) and said edge detector (21), for encoding 
said edge image and said reduced image. 

71. The digital image encoder of daim 70, wherein said smoothing filters (20-m) are coupled in cascade to 
said input terminal (10). 

72. The digital image encoder of claim 70^ wherein said smoothing filters (20-m) are low-pass filters with even 
symmetry. 

73. Thedigital tmageencoderofclaim70, wherein said edge detector (21) detects variations in rate of change 
of said pixel values from pixel to pixel. 

74. The digital image encoder of claim 70, wherein said encoding section (22) comprises: . 

an edge image encoder (24) coupled to said edge detector (21), for encoding said edge image to 
generate edge information; 

a low-frequency image encoder (25) coupled to said down sampler (23), for encoding said reduced 
image to generate low-frequency information; and 

a multiplexer (26), coupled to said edge image enccider(24) and said low-frequency image encoder 
(25), for multiplexing said low-frequency information and said edge information. 

75. The digital image encoder of claim 70, wherein said digitized image is a moving image consisting of suc- 
cessive frames, and said encoding section (65) comprises: 

a first memory (72) for storing a previous edge image, pertaining to a previous frame among said 
frames; 

a first motion estimator (66) coupled to said selector (50) and said first memory (72)^ for detecting 
motion of said edge image with respect to said previous edge image, generating edge motion vectors, 
detecting differences between said edge image and said previous edge image over and above said mo- 
tion of said edge image, and generating an edge difference image; 

a second memory (76) for storing a previous reduced image, pertaining to said previous firame; 

and 

a second motion estimator (67) coupled to said down sampler (23) and said second memory (76), 
for detecting motion of said reduced image with respect to said previous reduced image, generating low^ 
frequency motion vectors, detecting differences between said reduced image and said previous reduced 
image over and above said motion of said reduced image, and generating a reduced difference image. 

76. The digital image encoder of claim 75, wherein: 

said first motion estimator (66) is coupled to said second motion estimator (67); and 
said first motion estimator (66) uses said low-frequency motion vectors to select a range of search 
for said edge motion vectors. 

77. The digital image encoder of claim 75, wherein: 

said first motion estimator (66) is coupled to said second motion estimator (67); and 
said second motion estimator (67) uses said edge motion vectors to select a range of search for 
said low-frequency motion vectors. 

78. The digital image encoder of claim 75, wherein said encoding section (22) also comprises: 

a first difference encoder (69) coupled to said first motion estimator (66), for encoding said edge 
difference image, thereby generating edge difference information; 

a first difference decoder (70) coupled to said first difference encoder (69), fordecoding said edge 
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difference information, thereby generating a decoded edge difference image; 

a first adder (71) for adding said decoded edge difference image to said previous edge image to 
generate a new edge image, and storing said new edge image in said first memory (72) for use as a pre- 
vious edge image in a subsequent frame among said frames; 

a second difference encoder (73) coupled to said second motion estimator (67), for encoding said 
reduced difference image, thereby generating low-frequency difference information; 

a second difference decoder (74) coupled to said second difference encoder (73), for decoding 
said low-frequency difference information, thereby generating ^ decoded reduced difference image; 

an adder (75) for adding said decoded reduced difference image to said prevbus reduced image 
.. to generate a new reduced difference image, and storing said new reduced difference image in said sec- 
ond memory (76) for use as a previous reduced image in said subsequent frame; 

a motion vector encoder (68) coupled to said first motion estimator (66) and said second motion 
estimator (67), for encoding said edge motion vectors and said low-frequency motion vectors to generate 
motion information; and 

a multiplexer (78) coupled to said motion vector encoder (68), said first difference encoder (69), 
and said second difference encoder (73), for mutttplexing said motion information, said edge difference 
infornr^tion, and said low-frequency information. 

79. A digital image decoder for reconstructing a digitized image from encoded edge information and low-fre- 
quency information, comprising: 

a decoding section (28) for decoding said edge information to generate a horizontal and vertical 
pair of edge images, and decoding said low-frequency information to generate a reduced image; 

an up sampler (35) coupled to said decoding section (28), for up-sampling said reduced image to 
generate a low-frequency image equal in size to said edge images; 

an edge synthesizer (34) coupled to said decoding section (28), for synthesizing a pair of high- 
frequency images by filtering said pair of edge images with an edge synthesis filter; and 

an inverse wavelet transform processor (36), coupled to said up sampler (35) and said edge syn- 
thesizer (34), for performing an inverse wavelet transform on said decoded low-:f requency image and said 
pair of high-frequency images, thereby obtaining said digitized Image. 

80. The digital image decoder of claim 79, wherein said edge synthesis filter has coefficients that are gen- 
erated by using a high-pass filter to filter a standard edge. 

81. The digital image decoder of claim 80, wherein said standard edge consists of a first part and a second 
part, with pixel values that vary at one constant rate in said first part and at a different constant rate in 
said second part. 

82. The digital image decoder of claim. 80, wherein said high-pass filter satisfies a complete reconstruction 
condition. 

83. The digital image decoder of claim 80, wherein said inverse wavelet transform processor (36) filters said 
pair of high-frequency inrtages by a conjugate of said high-pass filter. 

84. The digital image decoder of claim 80, wherein said high-pass filter has even symmetry. 

85. The digital image decoder of claim 80, wherein said high-pass filter has odd symmetry. 

86. The digital image decoder of claim 85, wherein said edge synthesizer (34) comprises: 

a pair of differential edge synthesizers (44, 45) for filtering respective edge images by a filter having 
coefficients equal to differences between consecutive coefficients of said edge synthesis filter, thereby 
generating a pair of differential high-frequency image; and 

a pair of accumulators (46, 47) for accumulating values in respective differential high-frequency 
images. 

87. A digital image decoder for reconstructing a digitized image from an encoded image consisting of edge 
information and low-frequency information, comprising: 

a decoding section for decoding said edge information to generate a pair of edge images, compris- 
ing a horizontal edge image and a vertical edge image, and decoding said low-frequency information to 
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generate a reduced image; 

an up sampler (35), coupled to said decoding section, for up-sampling said reduced image to gen- 
erate a decoded low-frequency image equal in size to said edge Images; 

a plurality of edge synthesizers (34-m), each coupled to said edge information decoder (31 ), having 
5 respective edge synthesis filters, for synthesizing a series of pairs of high-frequency images by filtering 

said horizontal edge image horizontally and said vertical edge image vertically with said edge synthesis 
filters; and 

an inverse wavelet transform processor (36), coupled to said up sampler (35) and said plurality of 
edge synthesizers (34-m), for performing an inverse wavelet transform on said decoded low-frequency 
10 image and said series of pairs of high-frequency images. 

88. the digital image decoder of claim 87, wherein said edge synthesis filters have coefficients generated 
by filtering a standard edge with respective high-pass filters having progressively decreasing cut-off fre- 
quencies. 

IS 

89. The digital image decoder of claim 88, wherein said standard edge consists of a first part and a second 
part, with pixel values that vary at one constant rate in said first part and at a different constant rate in 
said second part. 

20 90. The digital image decoder of claim 88, wherein said inverse wavelet transform processor (36) filters said 
pairs of high-frequency images by conjugate filters of respective high-pass filters. 

91. The digital image decoder of claim 88, wherein said high-pass filters have even symmetry. 

25 92. The digital image decoder of claim 88, wherein said high-pass filters have odd symmetry. 



93. The digital image decoder of claim 92, wherein each of said edge synthesizers (34-m) separately com- 
prises: 

a pair of differential edge synthesizers (44, 45) for filtering said horizontal edge image horizontally 
30 and said vertical edge image vertically by a filter having coefficients equal to differences between con- 

secutive coefficients of said edge synthesis filter, thereby generating a pair of differential high-frequenc^ 
images; and 

a pair of accumulators (46, 47) for accumulating values in respective differential high-frequency 
images. 

35 

94. The digital image decoder of claim 87, wherein said decoding section (28) comprises: 

an edge information decoder (31), for generating said pair of edge images by decoding said edge 
information; and 

a low-frequency information decoder (32), for generating said reduced image by decoding said low- 
40 frequency information. 

95. The digital image decoder of claim 87, wherein: 

said encoded image is an encoded moving image comprising successive frames; 
said edge information comprises edge motion information and edge difference information; 
45 said low-frequency information comprises low-frequency motion information and low-frequency 

difference information; and 

said decoding section (79) performs motion compensation. 

96. The digital image decoder of claim 95, wherein said decoding section (79) comprises: 

50 a third difference decoder (81) for decoding said edge difference information to obtain an edge 

difference image; 

a motion information decoder (82) for decoding said edge motion information and said low-frequen- 
cy motion information to obtain edge motion vectors and low-frequency motion vectors; 

a fourth difference decoder (83) for decoding said low-frequency difference information to obtain 
55 a reduced difference image; 

a third memory (85) for storing a previous edge image, pertaining to a previous frame among said 
frames; 

a first motion compensator (84) coupled to said third difference decoder (81), said motion infor- 
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mation decoder (82), and said third memory (85), for adding said edge difference image to said previous 
edge image according to said edge motion vectors to generate a new edge image, storing said new edge 
image in said third memory (85) for use as a previous edge image in a subsequent frame among said 
frames, and generating said pair of edge images; 

5 a fourth memory (87) for storing a previous reduced image pertaining to said previous frame; and 

a second motion compensator (86) coupled to said motion information decoder (82), said fourth 
difference decoder (83), and said fourth memory (87), for adding said reduced difference image to said 
previous reduced image according to said low-frequency motion vectors, thereby generating said reduced 
image, and storing said reduced image in said fourth memory (87) for use as a previous reduced image 

10 in said subsequent frame. 
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(54) Image encoding and decoding method and apparatus using edge systhesis and inverse 
wavelet transform 



(57) A digitized Image is encoded by detecting edg- 
es In the image, encoding the position and sharpness 
of the detected edges, filtering the image by a low-pass 
filter to generate a low-frequency image, and encoding 
the low-frequency image. A digitized image encoded in 
this way is reconstructed by generating a horizontal 



edge image and a vertical edge Image from the encoded 
edge position and sharpness Information, synthesizing 
a jDaIr of high-frequency images by filtering the horizon- 
tal and vertical edge images with an edge synthesis fil- 
ter, decoding the low-friequency image, and performing 
an inverse wavelet transform oh the decoded low-fre- 
quency Image and the high-frequency images. 
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